42 research outputs found
A Comparative Study on Textual Saliency of Styles from Eye Tracking, Annotations, and Language Models
There is growing interest in incorporating eye-tracking data and other
implicit measures of human language processing into natural language processing
(NLP) pipelines. The data from human language processing contain unique insight
into human linguistic understanding that could be exploited by language models.
However, many unanswered questions remain about the nature of this data and how
it can best be utilized in downstream NLP tasks. In this paper, we present
eyeStyliency, an eye-tracking dataset for human processing of stylistic text
(e.g., politeness). We develop a variety of methods to derive style saliency
scores over text using the collected eye dataset. We further investigate how
this saliency data compares to both human annotation methods and model-based
interpretability metrics. We find that while eye-tracking data is unique, it
also intersects with both human annotations and model-based importance scores,
providing a possible bridge between human- and machine-based perspectives. We
propose utilizing this type of data to evaluate the cognitive plausibility of
models that interpret style. Our eye-tracking data and processing code are
publicly available
Detecting and Explaining Causes From Text For a Time Series Event
Explaining underlying causes or effects about events is a challenging but
valuable task. We define a novel problem of generating explanations of a time
series event by (1) searching cause and effect relationships of the time series
with textual data and (2) constructing a connecting chain between them to
generate an explanation. To detect causal features from text, we propose a
novel method based on the Granger causality of time series between features
extracted from text such as N-grams, topics, sentiments, and their composition.
The generation of the sequence of causal entities requires a commonsense
causative knowledge base with efficient reasoning. To ensure good
interpretability and appropriate lexical usage we combine symbolic and neural
representations, using a neural reasoning algorithm trained on commonsense
causal tuples to predict the next cause step. Our quantitative and human
analysis show empirical evidence that our method successfully extracts
meaningful causality relationships between time series with textual features
and generates appropriate explanation between them.Comment: Accepted at EMNLP 201
"Is the Pope Catholic?" Applying Chain-of-Thought Reasoning to Understanding Conversational Implicatures
Conversational implicatures are pragmatic inferences that require listeners
to deduce the intended meaning conveyed by a speaker from their explicit
utterances. Although such inferential reasoning is fundamental to human
communication, recent research indicates that large language models struggle to
comprehend these implicatures as effectively as the average human. This paper
demonstrates that by incorporating Grice's Four Maxims into the model through
chain-of-thought prompting, we can significantly enhance its performance,
surpassing even the average human performance on this task
User or Labor: An Interaction Framework for Human-Machine Relationships in NLP
The bridging research between Human-Computer Interaction and Natural Language
Processing is developing quickly these years. However, there is still a lack of
formative guidelines to understand the human-machine interaction in the NLP
loop. When researchers crossing the two fields talk about humans, they may
imply a user or labor. Regarding a human as a user, the human is in control,
and the machine is used as a tool to achieve the human's goals. Considering a
human as a laborer, the machine is in control, and the human is used as a
resource to achieve the machine's goals. Through a systematic literature review
and thematic analysis, we present an interaction framework for understanding
human-machine relationships in NLP. In the framework, we propose four types of
human-machine interactions: Human-Teacher and Machine-Learner, Machine-Leading,
Human-Leading, and Human-Machine Collaborators. Our analysis shows that the
type of interaction is not fixed but can change across tasks as the
relationship between the human and the machine develops. We also discuss the
implications of this framework for the future of NLP and human-machine
relationships